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Abstract 



This paper reports data from the evaluation of the numeracy component of a 
long-running educational intervention, covering the period from 2001 to 2008. 
QuickSmart is both an intervention and research project operating in Australian 
schools. It is a structured intervention program designed for middle-school 
students (ages 10 to 13 years) with significant learning difficulties in basic literacy 
and numeracy. The program aims to increase fluency (automaticity) in the most 
basic skills that underpin proficient performance in reading and in mathematical 
calculation and problem solving. The guiding principle is that building fluency and 
confidence in basic skills enables students to devote much more cognitive effort 
to the higher-order processes involved in reading for meaning and in solving 
mathematical problems. The QuickSmart project includes a strong research 
component that investigates the effectiveness of the intervention through 
collection of targeted students’ pre-test and post-test scores on standardised 
achievement tests. These QuickSmart students’ scores are compared to the pre- 
test and post-test results of average-achieving comparison students who come 
from their same class groups. The data indicate that the intervention has 
significantly improved the numeracy performance of more than 2,000 students 
with learning difficulties from over 90 schools in Australia. Students’ average 
improvement has been of the order of two years’ growth as measured by effect 
size calculations. The improvement of QuickSmart students on standardized 
scores has also helped ‘narrow the gap’ between their performance and that of 
average-achieving students. The evaluation of this program across differing 
contexts indicates the importance of consolidating students’ basic academic skills 
and providing ongoing support to instructors through a well-designed 
professional development program. 

Schools in the United States, Australia and the UK share a common problem — 
too many of their students fail to achieve an adequate standard in basic 
numeracy. Evidence of this can be found in the United States within the data 
reported from the National Mathematics Advisory Panel Report (Gersten et al., 
2009), in Australia from the results from the National Assessment Program: 



2 




Literacy and Numeracy (NAPLAN) and the earlier nationwide ‘benchmarking’ 
assessments (Commonwealth of Australia, 2008; MCEETYA, 2009), and in 
Britain from regular surveys conducted with both school students and adults 
(e.g., Coben, 2003; OfSTED, 2005). It is now accepted generally that, for a 
variety of reasons, at least five to ten per cent of students have significant and 
ongoing difficulties in learning mathematics. Current teaching methods and in- 
class intervention strategies such as differentiating learning activities according 
to students’ abilities (e.g., Ferguson, 2009) do not appear to overcome the 
learning problems experienced by most of these students. Unfortunately, the 
achievement gap between their numeracy skills and the expected standard for 
their age group widens over time. As a result, the majority of these students lose 
confidence in their own ability to cope with basic mathematics and feel powerless 
to change the situation. In many cases, their problems are still evident when they 
become adults, often placing limitations on the types of employment they can 
enter (House of Commons Public Accounts Committee: UK, 2009). It has been 
stated that: 

No child should move into her or his teenage years, and on into adulthood, 
unable to read, write or work with numbers. Without these fundamental skills, 
our young people are too often denied the opportunity to move on to further or 
higher education or to find well paid jobs. They are also at much greater risk of 
social exclusion (DENI, 2008, p. i). 

Causes of learning difficulty 

There are many factors that contribute to learning difficulties in the numeracy 
domain, ranging from some that are intrinsic to the student and others that are 
clearly environmental. Among the intrinsic factors operating in some cases are 
visual or auditory perceptual difficulties, information processing deficits, poor 
attending behaviours, working memory problems, and lack of effective learning 
strategies (Geary, 2005; Lerner & Kline, 2006; Westwood, 2008). In addition, one 
or two students in every hundred may have a specific learning disability 
( dyscalculia ) that adversely affects their ability to acquire number skills and 
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concepts or to process quantitative data (Landerl, Bevan & Butterworth, 2004; 
Wadlington & Wadlington, 2008). The environmental factors that adversely affect 
learning of numeracy skills include ambivalent community attitudes toward the 
importance of learning mathematics, lack of family support and encouragement, 
and an absence of structured learning opportunities in the preschool years. 
Children from disadvantaged backgrounds tend to be over represented among 
students with poor numeracy skills. For example, in Australia it has been found 
that students in remote and rural areas and Indigenous students have some of 
the lowest numeracy scores in national testing programs (Doig, 2001 ; Graham, 
Bellert & Pegg, 2007). However, among the most important environmental 
influences on numeracy development is the quality and effectiveness of the 
instruction that students receive in school (Farkota, 2005; Martin, 2007; Pincott, 
2004). Unfortunately, the currently favoured teaching approaches for 
mathematics that focus primarily on open-ended investigation, activity, and 
problem solving are not always effective in building and reinforcing basic number 
knowledge and computational skills. This is particularly the case among students 
with learning difficulties. Research has consistently found that, regardless of the 
underlying causes of their difficulties, these students learn best through explicit 
and systematic instruction that provides ample opportunities for fundamental 
knowledge and skills to become firmly established through guided practice and 
corrective feedback (Ellis, 2005; Gersten, Jordan & Flojo, 2005; Rowe, 2006; 
Swanson, Hoskyn & Lee, 1 999). They do not benefit from being plunged into 
open-ended learning activities that lack clarity and structure. 

With these issues in mind, in 2001 a team from the University of New England’s 
National Centre for Science, Information and Communication Technology and 
Mathematics Education for Rural and Regional Australia (SiMERR) designed an 
intervention program - titled QuickSmart - to reverse the trend of ongoing poor 
academic performance for students who have been struggling at school for 
several years and who are caught in a cycle of continued failure (Graham, 

Bellert, & Pegg, 2007). QuickSmart targets students with learning difficulties in 
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the middle school years and focuses on increasing their fluency (automaticity) in 
basic literacy and numeracy skills. This paper synthesizes research data on the 
effectiveness of the QuickSmart Numeracy component, as revealed in evaluation 
data covering the period 2001 to 2008. 

Key features of QuickSmart numeracy intervention 

QuickSmart is a teacher- or teacher aide-directed program that operates for 3 x 
30-minute lessons per week over a period of 30 weeks, usually spanning 3 
school terms. Students participate in the sessions in pairs and are taught in a 
withdrawal setting, not in the mainstream classroom. They are taught to develop 
effective strategy use and they participate in targeted practice activities. 
QuickSmart students spend considerable lesson time becoming ‘quicker’ at 
recalling number facts and performing simple calculations, and ‘smarter’ in 
strategy use. Both structured and incidental strategy instruction are important 
features of numeracy lessons, with the aim of moving students on from relying on 
slow and error-prone strategies (especially count-by-one strategies) to the use of 
more sophisticated and efficient strategies and automatic recall. Focusing on 
various domains in numeracy, the program enables instructors to plan instruction 
that meets individual students’ learning needs and also provides students with 
opportunities to self-monitor and to receive immediate, formative feedback. 

The content of QuickSmart Numeracy covers (but it is not limited to) addition, 
subtraction, multiplication and division facts, and triple addition tasks such as 7 + 
4+3 (where a quick and effective mental strategy is to recognise instantly that 7 
+ 3 makes 10, and then add 4). The QuickSmart program emphasises the 
usefulness and relevance of facts and strategies to regular classroom activities. 
This feature of the program is essential for facilitating transfer of learning to other 
settings. In relation to this point, it is also important to acknowledge that once 
students’ recall of basic facts and performance of basic tasks becomes truly 
automatic, they cannot help but have these facts and skills available for use in 
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other settings and on more complex tasks. It is important that middle-school 
students have ready access to such prerequisite academic skills that enable 
them to engage fully with challenging academic work. 

QuickSmart learning and teaching strategies are drawn directly from research 
evidence identifying effective methods for students with learning difficulties (e.g., 
Bryant et al., 2008; Gersten, Jordan & Flojo, 2005; Rowe, 2006; Swanson, 2000). 
These include explicit strategy instruction, modeling, discussion, questioning, 
feedback, guided and independent practice, and frequent reviews. Each lesson 
involves brief revision of work covered in the previous session, a number of 
guided practice activities featuring overt self-talk, discussion and practice of 
memory and retrieval strategies, and games and worksheet activities followed by 
timed and independent practice activities. 

Ongoing, formative assessment is an integral part of the QuickSmart intervention 
program and ensures that the learning program is tailored to extend the existing 
knowledge and skills of individual learners. Most lessons conclude with an 
assessment using the computer-based Cognitive Aptitude Assessment System 
(CAAS) to provide the student and the instructor with information about the 
accuracy and speed of recall of basic facts. This software was developed at the 
Laboratory for the Assessment and Training of Academic Skills (LATAS) at the 
University of Massachusetts (Royer &Tronsky, 1998; Royer, Tronsky, & Chan, 
1999). 

The main principles underpinning the QuickSmart program are that it: 

• is designed to improve students’ information retrieval times; 

• frees working-memory capacity from an excessive focus on routine tasks; 

• fosters automaticity in basic tasks; 

• utilises explicit teaching based on understanding and deliberate practice, 
not on rote learning; 

• has time (as well as accuracy) as a dimension of learning; 
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• integrates assessment tasks into each lesson with a focus on individual 
improvement; 

• maximises student on-task time in a structured but flexible lesson format; 

• provides extensive materials including teaching resources, speedsheets, 
flashcards; and 

• incorporates the Cognitive Aptitude Assessment System (CAAS) to 
assess students’ speed and accuracy on basic mathematics tasks. 

The following principles have guided both the development and scaling up of the 
QuickSmart intervention: 

• Programs designed to address the learning needs of low-achieving middle- 
school students should be intense, of significant enough duration to make a 
difference, and conducted in small class instructional settings. 

• An extensive professional learning program for teachers, teacher aides and 
executive members of schools and education jurisdictions should be an 
important component of any sustainable instructional intervention. 

• Improving the skill base of teacher aides should be a focus of attention for all 
support programs, especially those in rural and remote areas or schools that 
are difficult to staff and where teaching staff mobility is a significant factor. 

• To ensure sustainability, National, State, regional and school level 
stakeholders need to coordinate their efforts and collaborate to ensure the 
fidelity of the program, and the viability of its implementation and scaling up 
processes. 

The professional development program accompanying QuickSmart is focused on 
supporting teachers to understand and provide: 

• effective instruction that maximises student on-task time and opportunities for 
meaningful feedback; and provides learning scaffolds to ensure students 
experience improvement and success; 
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• deliberate practice that is integral to every lesson, allows for success and is 
focused on providing targeted feedback to improve learning; 

• guided and independent timed practice activities; 

• strategy instruction and concept development; 

• confidence in their students by encouraging a ‘can do’ attitude; 

• appropriate teacher and peer modeling; and 

• motivational academic activities that develop fluent performance. 

A teacher coordinator is required in each participating school. Professional 
learning is offered during school time on six professional learning days. Support 
is then ongoing, as required. An introductory session for principals is also 
provided. 

The implementation of QuickSmart in Australia has been supported by research 
grants from the Australian Research Council, the Federal Government, project 
funds from SiMERR, and extensive cash and in-kind support from the Northern 
Territory and New South Wales. Since 2001 , when the intervention was first 
introduced on a small scale in New South Wales, QuickSmart has been 
implemented on an increasingly expansive scale. In 2008 the program had 
extended to more schools in New South Wales and the Northern Territory, and 
was being introduced in South Australia, Victoria, and the Australian Capital 
Territory. The number of schools involved so far in 2009 is 148 schools. 

QuickSmart is one of the main intervention programs recommended for adoption 
by schools in the National Partnership on Literacy and Numeracy, a joint initiative 
of the Australian and New South Wales governments (State of New South 
Wales, 2009). Many of the resources required to implement the QuickSmart 
programs are provided in the QuickSmart Numeracy and Literacy Kits. The Kits 
for both programs include administrative and organisational information, 
learning/teaching resources, and a QuickSmart DVD. In addition, QuickSmart 
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provides the Cognitive Aptitude Assessment System (CAAS) software for 
ongoing assessment throughout the duration of the QuickSmart programs. 

Details of QuickSmart can be found online 

at: http://www. une.edu.au/simerr/quicksmart/pages/index.php 

Evaluating QuickSmart 

A critical research aspect of the implementation of QuickSmart has been the 
attention paid to the ongoing intensive evaluation of the program. Over the period 
2001 to 2008 systematic data collection has accrued substantial empirical 
evidence regarding the value and applicability of the QuickSmart Numeracy 
program (Graham, Bellert, & Pegg, 2001 ; Graham, Bellert, Thomas, & Pegg, 
2007; Pegg, Graham, & Bellert, 2005). The accumulation of such evidence over 
time from multiple jurisdictions across a range of geographic and socio-economic 
contexts is, we believe, a more powerful evaluation procedure for establishing 
the veracity, usefulness, effectiveness and sustainability of the program than any 
single controlled experimental study. 

The QuickSmart project uses a quasi-experimental research design involving 
collecting and analysing pre-test and post-test data from two groups of students: 
(i) the ‘QuickSmart Students’, who participate in the numeracy and/or literacy 
intervention programs; and (ii) ‘Comparison Students’ who do not participate in 
the intervention programs. These comparison students are average achievers in 
mainstream mathematics, are the same age as the QuickSmart students, and 
are drawn from the same schools. They complete the selected CAAS sub-tests in 
numeracy at the beginning and the end of the intervention period and also 
participate in the standardised testing sessions. Pre-test and post-test data are 
collected by school-based QuickSmart co-ordinators for both sets of students 
using the CAAS test results and the independent state-wide or standardised 
achievement tests results. These data help to quantify ways that QuickSmart 
narrows the achievement gap for low-achieving students, and serves to isolate 
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any effects attributable to the instructional program. Interviews and surveys of 
students, parents, teachers, and principals involved in QuickSmart have also 
yielded important qualitative data on the program’s effectiveness. 

All data from schools other than those in the Northern T erritory are sent to the 
SiMERR National Centre at the University of New England, where they are 
transferred to electronic spreadsheets or word processing programs. In the case 
of schools in the Northern Territory, officers from the Department of Education 
and Training have (since 2006) independently collected data from participating 
schools and then undertake the analysis before forwarding the results to 
SiMERR. 



Participants 

Table 1 summarises the various cohorts of middle-school students with 
mathematics learning difficulties who have participated in QuickSmart since 
2001, together with the numbers of normally-achieving students used for 
comparison purposes in each year. 



Table 1: Summary of QuickSmart data for all Regions/School Sectors, 2001 - 2008 



Year 


Schools 


QS Students 


Comparison 

Students 


All Students 
{QS + Comp) 


Indigenous/NESB 

Students 


2001 


2 


20 


13 


33 


N/A 


2002 


3 


18 


0 


18 


N/A 


2003 


10 


63 


40 


103 


N/A 


2004 


8 


72 


43 


115 


N/A 


2005 


13 


130 


141 


271 


N/A 


2006 


19 


245 


118 


363 


116 


2007 


55 


780 


269 


1049 


215 


2008 


91 


772 


206 


978 


268 


TOTAL 


201 


2100 


830 


2930 


599 



Note: QS = QuickSmart. NESB = Non English speaking background 



Analysis of data 
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Analysis of quantitative data involved comparing the standardised tests and 
state-wide assessments results, followed by analyses of the CAAS measures 
using MANOVA and ANOVA statistics and a follow-up using step-wise 
regression. Effect Size calculations were completed where appropriate for all the 
available QuickSmart data. The major rationale for utilising MANOVA was to 
examine whether the QuickSmart intervention program was associated with 
differences in mean scores obtained on a considerable number of dependent 
variables. The rationale for using step-wise regression was that this procedure 
makes no assumptions about the relative importance of variables and instead 
selects variables for entry into the model in an order that reflects the extent to 
which they explain shared variance, and the extent to which this sharing is 
statistically significant. Conversely, step-wise regression excludes variables from 
the model that do not explain a sufficient portion of the shared variance. A 
feature of this procedure is that when two variables overlap in their capacity to 
account for the shared variance, the variable with the greater capacity to do so is 
entered. 

In addition to these analyses across the whole set of the QuickSmart data from 
2001 to 2008, Effect Sizes were also calculated for each region or Territory 
where QuickSmart has been implemented since the program began. Effect Sizes 
were used here to quantify the effectiveness of interventions relative to 
comparison groups. Discussion of Effect Sizes enables researchers to move 
beyond the simplistic, ‘Does it work or not?’ to the more useful, ‘How well does it 
work in a range of contexts?’ Based on the work of Hattie (2009) an insignificant 
effect size is around 0.1, an average effect size is around 0.3, important effect 
sizes begin above 0.4, and significantly important effect sizes occur above 0.6. 
Data on Effect Sizes in this program are reported later in the paper. 

Standardised tests 

The main yardstick used for assessing progress over time was the Progressive 
Achievement Test (PAT) in mathematics (ACER, 2005). It is important to note 
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that over the eight-year span of this analysis, versions of the PAT used in 
schools have varied. Therefore, where possible, raw scores have been 
transformed to scale scores (PATM) which are consistent across all versions of 
the PAT tests. 

Cognitive Aptitude Assessment System (CAAS) 

Measuring changes in accuracy and automaticity of basic academic skills and the 
recall of basic facts is an integral part of this research. Upon admission to the 
QuickSmart program students complete an assessment process consisting of 
CAAS tasks that measure the speed and accuracy of hierarchically arranged 
basic mathematics tasks. Speed is measured using tasks that involve the 
appearance of a stimulus on the computer screen followed by the student 
responding into a microphone. The CAAS provides highly accurate measures of 
how rapidly students complete the tasks and an assessor then scores the 
response for accuracy. The CAAS assessment process involves completion of 
tasks that measure number identification, and a range of addition, subtraction, 
multiplication and division tasks. The CAAS data comprise results from five tests 
based on sets of twenty randomly generated individual questions. The five tests 
are referred to as: number naming, addition, subtraction, multiplication, and 
division. In each test, scores are collected for both percentage accuracy 
(accuracy) and the average time taken for a response to each question (speed). 

Results 

Looking first at the evidence obtained from the pre- and post-testing using PAT, 
the following data can be reported. Difference scores based on the available raw 
scores from PAT in mathematics indicated that overall the average difference 
score for the 1354 QuickSmart students was 5.63 (SD = 6.84) compared to an 
average difference score of 3.78 (SD = 7.62) for the 530 comparison students. 

Using available PATM data, the descriptive statistics indicate that the difference 
scores for 573 QuickSmart Numeracy students averaged 6.70 (SD = 7.50), while 
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difference scores for the comparison students averaged 3.67 (SD = 7.04). 
Importantly, the gain for 120 Indigenous students with PATM difference scores 
averaged an impressive 7.07 points (SD = 8.66). Table 2 displays mean 
difference scores for data from the Progressive Achievement Tests in 
Mathematics. 

Table 2: Means and Standard Deviations for PAT Difference Scores by QuickSmart 
and Comparison Students 



Group 


Mean 


QuickSmart 

SD 


Mean 


Comparison 

SD 


Raw Score Difference 


5.63 


6.84 


3.78 


7.62 


PATM Difference 


6.70 


7.50 


3.67 


7.04 



A between-groups multivariate analysis of variance (MANOVA) was conducted to 
determine the effect of group membership {QuickSmart or Comparison) on four 
dependent variables (DVs), (i.e., difference scores based on PAT pre-test and 
composite score data). Significant effects were found for group membership on 
the multivariate dependent measures, Wilks’ A= 0.71, F(4,671) = 67.75, p<0.001. 
Univariate tests (Analysis of variance (ANOVA)) provide an indication of whether 
specific independent variables (IVs) are significantly associated with specific 
DVs. The main effect for treatment condition was statistically significant for 
difference scores based on the raw PAT scores (F(1 ,674)=35.19, p<0.001), 

PATM scores across all versions of the PAT tests (F(1,674)=16.42, p<0.001), 
and PAT stanine scores (F(1 ,674)=14.33, p<0.001). 

Table 3 presents the means and standard deviations for the PAT mathematics 
tests. 
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Table 3: Means and Standard Deviations for each PAT Mathematics Test 
Dependent Variable for QuickSmart and Comparison Students 



Group 


M 


QuickSmart 

SD 


M 


Comparison 

SD 


RawScore Dif 
Cohen’s d = .59 


4.93 


5.14 


2.20 


4.30 


PATM Dif 


6.61 


7.41 


3.85 


7.18 


PctIRnkg Dif 


11.87 


14.72 


9.53 


16.74 


Stanine Dif 


1.04 


1.15 


.633 


1.16 



Looking next at the data obtained from the CAAS assessments, Table 4 
summarises the MANOVA and ANOVA statistics for all CAAS assessments in 
mathematics. 



Table 4: Influence of mathematics intervention group on mathematics outcomes 



Dependent Variable 


df 


Df error 


MS 


F 


Sig. 


Numerical identification speed 


1 


582 


2.361 


9.379 


★ ★ 


Numerical identification accuracy 


1 


582 


15.899 


0.688 


NS 


Addition speed 


1 


1020 


93.537 


40.958 


★ ★★ 


Addition accuracy 


1 


1020 


2216.907 


21.527 




Subtraction speed 


1 


967 


135.264 


49.685 




Subtraction accuracy 


1 


967 


6233.29 


44.278 




Multiplication speed 


1 


953 


256.718 


40.564 




Multiplication accuracy 


1 


943 


17388.33 


58.71 




Division speed 


1 


914 


379.903 


60.969 




Division accuracy 


1 


914 


24344.567 


78.079 





**p<0.01, ***p<0.001 



Significant outcomes were obtained for group membership {QuickSmart versus 
Comparison students) on all the CAAS measures except for the accurate 
identification of numerals, a very simple subtest that did not differentiate between 
the students because of the high accuracy levels of both groups. This group of 
findings is easily interpretable and important in terms of the design of the 
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QuickSmart Numeracy program and its focus on the accuracy of basic facts and 
speed of recall. 

The major outcome of the step-wise regression was that the QuickSmart 
numeracy intervention predicted all of the mathematics change scores. This 
means that the effect of the QuickSmart program was strong across both the 
standardized and CAAS measures. 

Turning now to the Effect Size analyses, the data here were obtained from 
schools in various regions of NSW and from the Northern Territory. It is useful to 
be reminded again that in educational research Effect Sizes below 0.2 are 
considered weak because an appropriate and ‘normal’ range of growth over an 
academic year for a student cohort would be within the range of 0.2 to 0.4. 
Effect-size scores of 0.4 to 0.6 are considered strong, those between 0.6 and 0.8 
are considered very strong, while those above 0.8 represent substantial 
improvement of the order of approximately three years’ growth. 

The official report evaluating Quicksmart (NCSiMERRA, 2009) contains Effect 
Size data tables for many separate regions participating in the program and for 
different years of involvement. The scope and length of this paper do not allow 
for reproduction of all these tables, so for convenience Effect Size results are 
summarised here. In this longitudinal study, the Effect Sizes obtained across 
schools and jurisdictions are remarkably consistent, ranging from 0.49 to 0.80, 
with greater effects evident for the QuickSmart students over the comparison 
group’s performance. Secondly, across the board the Effect Sizes based on the 
scores of the QuickSmart students are well above the expected yearly average 
growth of around 0.3. For example: 

• In the Northern Territory during 2006, 2007, and 2008 the effect size growth 
of many hundreds of QuickSmart students based on state-wide tests was 
0.68, 0.60 and 0.78, respectively compared to a considerably lower effect size 
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of approximately 0.3 or less calculated for the average-performing 
comparison cohorts. 

• Students from the eight schools which participated in QuickSmart in the NSW 
North Coast Region in 2007 recorded an effect size of 0.75 on the ACER PAT 
tests. In contrast, the comparison cohort’s effect size value was calculated to 
be 0.19. The improvement of the QuickSmart students represents 
approximately three years’ growth over the course of a single year. This result 
improved further in 2008 with an effect size of 0.801 calculated for the 
QuickSmart sample of 238 low-achieving students. 

• An analysis by an independent statistician of the large data-sets of ACER 
PATM scores from several hundred NSW students found that the effect sizes 
for QuickSmart students ranged from 0.59 to 0.69, with the latter figure 
representing those students who completed the full thirty weeks of instruction. 

Some Effect Size comparisons for specific regions in NSW are summarised in 

Table 5. 



Table 5: Effect Sizes: NSW Regions 2002 -2008 



Region/Cluster 


Pre-scores PATM 
(SD) 


Post-scores PATM 
(SD) 


Effect Size 
(Cohen’s d) 


Armidale 2002-2004 

QuickSmart students 


47.12 (8.00) 
n = 30 


52.14 (9.67) 
n = 28 


0.56 


Comparison students 


59 (10.29) 
n= 12 


61.56 (12.48) 
n = 9 


0.22 


Lismore 2002-2008 

QuickSmart students 


45.36 (8.73) 
n = 62 


51.74 (7.12) 
n = 64 


0.80 


Comparison students 


54.94 (7.46) 
n= 17 


53.63 (7.87) 
n = 16 


-0.17 


North Coast -2008 

QuickSmart students 


44.05 (8.24) 
n = 375 


50.92 (9.53) 
n = 334 


0.77 


Comparison students 


55.29 (7.93) 
n = 89 


58.53 (10.31) 
n = 88 


0.33 
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Table 6 summarises Effect Size data from the Northern Territory, as calculated 
independently by staff of the NT Department for Education and Training. This 
table contrasts the effects of QuickSmart Numeracy on Indigenous and Non- 
Indigenous students. The Northern Territory is home to a large number of 
Indigenous students, and their poor literacy and numeracy standards are a 
matter of ongoing concern. It seems, however, that the approach used in 
QuickSmart Numeracy is highly appropriate for raising the numeracy standards 
of these students (Effect Size 0.76). 



Table 6: Effect Sizes: Northern Territory 2008 





Effect Size 


Confidence 

Interval 


Significance 
(wrt Comparison) 


QuickSmart- All 


0.78 


±0.07 


p = 0.0029 


QuickSmart- Indigenous students 


0.76 


±0.09 


p = 0.01 74 


QuickSmart - Non-lndigenous 
students 


0.88 


±0.12 


p = 0.0033 



It can from the Effect Size evidence above that QuickSmart Numeracy has 
proved to be a highly effective program for increasing and accelerating the basic 
number skills for a wide variety of students with learning difficulties in a wide 
variety of settings. 

Finally, the qualitative evidence obtained from interviews and surveys involving 
students, parents, teachers, and principals have indicated great support and 
enthusiasm for QuickSmart. Three typical comments are included below. The 
official report (NCSiMERRA, 2009), with sets of evidence for over 2,000 students 
and many hundreds of teachers and parents, contains very many more 
examples. 

A female student’s comment: 

I know my times tables better than I did. I’ve improved my speed by finding 
short ways of doing the number facts. And I know about denominators and 
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numerators. And how to change things into a decimal or a percentage and how 
to put things in the right groups. (2003, New England Girls’ School, Armidale, 
numeracy student) 

A parent’s comment: 

QuickSmart has had a huge effect on our daughter’s performance at school. 

Most notably the Basic Skills [Test] results. In Year 3 she was in the bottom 
30% of the state. This year, in Year 5, she was in the top 30%. (2005, St 
Francis Xavier Woolgoolga) 

An administrator’s comment: 

My experiences in viewing QuickSmart in action in the schools in New England 
are all positive. I have found many students, who were previously disengaged 
with mathematical activities, totally engaged in the activities and process that 
form a major part of the intervention... Independent research in the New England 
region indicated that students, including Aboriginal students, make quick gains in 
their ability and confidence to use mathematics. (Mr Des Gorman A/General 
Manager, Learning and Development, NSW DET) 

Conclusion 

The learning difficulties experienced by many middle-school students are 
persistent and resistant to change unless they are provided with sustained and 
intensely focused personalised instruction. QuickSmart is an intervention 
program that targets, with small group instruction, those students in the lower 
30% of the achievement spectrum. Analysis of data from a wide range of settings 
has identified impressive statistically significant gains in terms of probability 
measures and Effect Sizes that mirror the qualitative improvements reported by 
teachers, instructors and parents. Students who complete the QuickSmart 
program show general, sustained improvements in independent learning, self- 
regulation, metacognition and self-esteem. The strong quantitative and 
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qualitative evidence base reported here confirms that QuickSmart helps to 

‘narrow the gap’ for low-achieving middle-school students. 
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